feat: inject OpenRouter provider.only for BYOK / no-fallback routing - #19
Open
echarrod wants to merge 1 commit into
Open
feat: inject OpenRouter provider.only for BYOK / no-fallback routing#19echarrod wants to merge 1 commit into
echarrod wants to merge 1 commit into
Conversation
Add OPENROUTER_PROVIDER_ONLY env var (comma-separated provider slugs)
that the proxy injects as {"provider": {"only": [...]}} into every
request body when the detected provider is OpenRouter. This pins
routing to specific upstream provider(s) so requests use the user's
BYOK key and fail hard instead of silently falling back to a
credit-billed provider (e.g. Bedrock/Vertex).
allow_fallback is intentionally omitted: OpenRouter rejects it as an
unrecognized key inside provider, and only already implies fail-hard.
Note: provider.only constrains the provider slug only. It does not
prevent within-slug fallback from a BYOK endpoint to OpenRouter's own
pool for the same slug; that still requires the "Always use for this
provider" toggle on a Prioritized BYOK key.
Closes nielspeter#8
Co-Authored-By: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds an
OPENROUTER_PROVIDER_ONLYenv var (comma-separated provider slugs) that the proxy injects as{"provider": {"only": [...]}}into every request body when the detected provider is OpenRouter. This pins routing to specific upstream provider(s) so requests use the user's BYOK key and fail hard instead of silently falling back to a credit-billed provider (e.g. Anthropic → Bedrock, Google → Vertex).Closes #8 (and the feature requested in #18).
Why
When using OpenRouter with a BYOK key, OpenRouter can fall back to a different provider serving the same model if the primary provider errors or is rate-limited. That fallback bills OpenRouter credits even when the user only wants to use their own provider key. OpenRouter's documented fix is to send a
providerfield in the body, but the proxy had no way to configure it. The proxy already builds and marshals the OpenAI request body ininternal/converter/converter.go, so it is the natural place to inject this.Changes
pkg/models/types.go— addProvider map[string]interface{} json:"provider,omitempty"toOpenAIRequest.internal/config/config.go— addOpenRouterProviderOnly []string, parsed fromOPENROUTER_PROVIDER_ONLY(trimmed, empties dropped, nil when unset).internal/converter/converter.go— in the OpenRouter path, setopenaiReq.Provider = {"only": cfg.OpenRouterProviderOnly}when configured. Applied to both streaming and non-streaming requests (not gated on the streaming branch).Design notes
allow_fallbackis intentionally omitted. OpenRouter returns400 provider: Unrecognized key: "allow_fallback"if it is included insideprovider, andonlyalready implies fail-hard (no fallback) per OpenRouter's docs.provider.onlyalone gives them zero credit usage.Testing
go buildclean;go test ./...all packages pass (config, converter, daemon, server).OPENROUTER_PROVIDER_ONLY=google-ai-studioandANTHROPIC_DEFAULT_OPUS_MODEL=google/gemini-2.5-pro:"provider": {"only": ["google-ai-studio"]}.is_byok: false) within the same slug and billed credits.provider.onlyblocked the cross-provider path but could not block this; the BYOK toggle is required for that.Usage
In
~/.claude/proxy.env(or.env):Omit the var to leave routing behaviour unchanged.